Controller device

ABSTRACT

A controller device configured to be worn on a hand of a user includes a plurality of sensors that, in a state in which the user wears the controller device on the hand, are disposed along a virtual curved surface being projected toward a position of a thumb of the user and detect the position of the thumb of the user.

TECHNICAL FIELD

The present invention relates to a controller device used while beingworn on a hand of a user, an information processing device, aninformation processing method, and a program.

BACKGROUND ART

In technology such as virtual reality, for example, an operation inputfrom a user is sometimes received through a gesture such as a movementof a finger of the user. As one of techniques for recognizing such amovement of a finger of the user, a method has been considered whichallows a controller device to be worn on a hand of the user and allows asensor disposed on a surface of the controller device to detect a stateof the finger of the user.

[SUMMARY] [TECHNICAL PROBLEM]

Among fingers of a human hand, a thumb has a wider movable range thanother four fingers, and thus, detecting an accurate position of thethumb requires a sensor having a wide detection-target range. Inparticular, attempting to accurately detect the position of the thumb,by using a sensor having a relatively small detection range, a largenumber of the sensors are required.

The present invention has been made in view of the above situation, andan object of the present invention is to provide a controller device, aninformation processing device, an information processing method, and aprogram that are capable of efficiently identifying a state of the thumbof a hand of a user using a limited number of sensors.

Solution to Problem

A controller device according to an aspect of the present invention is acontroller device configured to be worn on a hand of a user, andincludes a plurality of sensors that, in a state in which the user wearsthe controller device on the hand, are disposed along a virtual curvedsurface being projected toward a position of a thumb of the user anddetect the position of the thumb of the user.

An information processing device according to an aspect of the presentinvention includes an acquisition section that acquires, from acontroller device configured to be worn on a hand of a user andincluding a plurality of sensors that, in a state in which the userwears the controller device on the hand, are disposed along a virtualcurved surface being projected toward a thumb of the user and detect aposition of the thumb of the user, detection values of the plurality ofindividual sensors, and a state identification section that identifies astate of the thumb of the user by calculating a rotation angle of atleast one or more joints of the thumb of the user on the basis of theacquired detection values.

An information processing device according to an aspect of the presentinvention is an information processing device for controlling acontroller device configured to be worn on a hand of a user andincluding a sensor that detects a position of a finger of the userwithin a predetermined detection range and an operation button operatedby the finger, and includes an acquisition section that acquires adetection result of the sensor, and a state identification section thatidentifies a state of the finger on the basis of the detection result ofthe sensor. When the operation button is pushed, the stateidentification section rejects the detection result of the sensor andidentifies the state of the finger on the basis of an operation amountdetected by the operation button.

An information processing method according to an aspect of the presentinvention includes the steps of acquiring, from a controller deviceconfigured to be worn on a hand of a user and including a plurality ofsensors that, in a state in which the user wears the controller deviceon the hand, are disposed along a virtual curved surface being projectedtoward a thumb of the user and detect a position of the thumb of theuser, detection values of the plurality of individual sensors, andidentifying a state of the thumb of the user by calculating a rotationangle of at least one or more joints of the thumb on the basis of theacquired detection values.

A program according to an aspect of the present invention is a programfor causing a computer to execute the steps of acquiring, from acontroller device configured to be worn on a hand of a user andincluding a plurality of sensors that, in a state in which the userwears the controller device on the hand, are disposed along a virtualcurved surface being projected toward a thumb of the user and detect aposition of the thumb of the user, detection values of the plurality ofindividual sensors, and identifying a state of the thumb of the user bycalculating a rotation angle of at least one or more joints of the thumbon the basis of the acquired detection values. The program may beprovided in such a way as to be stored in a non-transitorycomputer-readable information storage medium.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a configuration block diagram of an information processingsystem including a control device according to an embodiment of thepresent invention.

FIG. 2 is a perspective view of a controller device illustrating anexample of an external appearance thereof.

FIG. 3 is a right side view of the controller device illustrating anexample of an external appearance thereof.

FIG. 4 is a rear side view of the controller device illustrating anexample of an external appearance thereof.

FIG. 5 is a function block diagram of an information processing device.

FIG. 6 is a diagram schematically illustrating joints of a finger of ahand.

DESCRIPTION OF EMBODIMENT

Hereinafter, an embodiment of the present invention will be described indetail on the basis of the drawings.

FIG. 1 is a configuration block diagram illustrating an overview of aninformation processing system 1 including a controller device accordingto an embodiment of the present invention. As illustrated in FIG. 1, theinformation processing system 1 includes an information processingdevice 10, a controller device 30, and a head-mounted type displaydevice 40. In the present embodiment, the information processing device10 functions as a control device for controlling the controller device30.

The information processing device 10 is a device for processing resultsof detections by the controller device 30 and may be, for example, ahome game machine, a handheld game machine, a personal computer, asmartphone, a tablet, or the like. As illustrated in FIG. 1, theinformation processing device 10 includes a control unit 11, a storageunit 12, and an interface unit 13.

The control unit 11 includes at least one processor such as a CPU(Central Processing Unit) and performs various information processes byexecuting programs stored in the storage unit 12. Note that a specificexample process performed by the control unit 11 in the presentembodiment will be described later. The storage unit 12 includes atleast one memory device such as a RAM (Random Access Memory) and storesthe program executed by the control unit 11 and data processed by theprogram.

The interface unit 13 is an interface for data communication with thecontroller device 30 and the display device 40. The informationprocessing device 10 is coupled to each of the controller device 30 andthe display device 40 in a wired or wireless manner via the interfaceunit 13. Specifically, the interface unit 13 includes a multimediainterface such as HDMI (registered trademark) (High-DefinitionMultimedia Interface) to transmit video data supplied by the informationprocessing device 10 to the display device 40. Further, the interfaceunit 13 includes a data communication interface such as a USB (UniversalSerial Bus) to receive the results of the detections by the controllerdevice 30.

The controller device 30 is a device worn on a hand of a user and usedfor detecting the movement of each of one or more fingers of the hand.FIGS. 2, 3, and 4 each illustrates an example of an external appearanceof the controller device 30. FIG. 2 is a perspective view of thecontroller device 30 as viewed from slightly above illustrating itsfront side (a thumb side at the time when the user is wearing thecontroller device 30); FIG. 3 is a right side view; and FIG. 4 is a rearside view. As illustrated in these figures, the controller device 30includes a main body 31 and a fixture 32, and the user wears thecontroller device 30 on his or her hand by passing his or her palmthrough the fixture 32. Here, it is supposed that the user wears thecontroller device 30 on his or her left hand, and thus, the fixture 32is attached to the left side of the main body 31 as viewed toward thefront. Here, the fixture 32 may be configured by a belt or the likewhose length is adjustable according to the size of the hand of theuser. Note that although it is supposed here that the user wears thecontroller device 30 on his or her left hand, the user may wear thecontroller device 30 on each of his or her both hands.

A plurality of proximity sensors 33 are disposed on a surface of thehousing of the main body 31 to detect the movement of the finger of theuser. Each of the proximity sensors 33 is a sensor, such as an infraredsensor or an electrostatic sensor, which is capable of measuring adistance to an object existing within a detection range. The informationprocessing device 10 identifies the position of a user's finger targetedfor detection, by using detection results of the proximity sensors 33.In the present embodiment, it is assumed that the user's finger targetedfor the detections by the proximity sensors 33 is the thumb of a hand onwhich the user wears the controller device 30.

In the present embodiment, a sensor region 34 having a curved-surfaceshape and facing the user's thumb is disposed on the surface of thehousing of the main body 31. As illustrated in FIGS. 2 and 3, the sensorregion 34 is disposed obliquely upward at the front side of the mainbody 31, and the plurality of proximity sensors 33 are disposed insidethe sensor region 34. The sensor region 34 is disposed on a virtualcurved surface that is projected toward a user's thumb side. Thisvirtual curved surface has a shape approximate to a spherical surfaceshape in contact with the base of the user's thumb as viewed from thelateral side in a state in which the user wears the controller device30.

Further, the sensor region 34 is a region having an elongated curvedshape on the virtual curved surface. More specifically, the sensorregion 34 is a region having an approximately circular arc-shape along avirtual curved line that is projected toward a side opposite the baseside of the user's thumb (that is, an upper right-hand side in FIG. 2)in a state in which the user wears the controller device 30 on the hand.Further, the proximity sensors 33 are disposed so as to be arranged atapproximately equal intervals along the virtual curved line. As a resultof this configuration, the proximity sensors 33 are disposed along amovement trajectory of the tip of the user's thumb at the time when theuser rotates the thumb about a joint of the base of the thumb (that is,a CMC joint described later) in a state in which the user wearing thecontroller device 30 stretches the thumb. Disposing the proximitysensors 33 along such a curved line enables a relatively small number ofthe proximity sensors 33 to detect the position of the thumb even whenthe user moves his or her thumb over a wide range. In the presentembodiment, as a specific example, a configuration is employed in whichfour proximity sensors 33-1 to 33-4 are disposed inside the sensorregion 34. In this regard, however, the number of the proximity sensors33 is not limited to four and may be three or may be five or more.

Further, a trigger button 35 is disposed at a rear side of thecontroller device 30 (that is, at a side opposite the side at which theproximity sensors 33 are disposed). The trigger button 35 is a buttonfor receiving a push operation by an index finger of the user and isconfigured to be capable of detecting a push-in amount of the pushoperation. A proximity sensor 36 is disposed on a surface of the triggerbutton 35. The proximity sensor 36 may be also, similarly to theproximity sensors 33, an infrared sensor, an electrostatic sensor, orthe like and measures a distance to the user's index finger located nearthe trigger button 35. That is, the controller device 30 detects themovement of the user's thumb, by using the proximity sensors 33 anddetects the movement of the user's index finger, by using the proximitysensor 36 and the trigger button 35. Note that one or more sensors fordetecting the state of a finger other than the thumb and the indexfinger may be further disposed on the controller device 30.

Further, the controller device 30 incorporates a control circuit 37. Thecontrol circuit 37 transmits a signal indicating the contents of theresults of the detections by the proximity sensor 36 and the pluralityof individual proximity sensors 33 to the information processing device10. Further, the control circuit 37 transmits a signal indicating thepush-in amount of the trigger button 35 to the information processingdevice 10.

The display device 40 is a video display device used while being mountedon the head of the user and displays a video corresponding to a videosignal transmitted from the information processing device 10 to allowthe user to browse the video.

Next, functions implemented by the information processing device 10 willbe described using FIG. 5. As illustrated in FIG. 5, the informationprocessing device 10 functionally includes a sensor value acquisitionsection 51, a finger state identification section 52, and a videodisplay control section 53. These functions are implemented by allowingthe control unit 11 to execute a program stored in the storage unit 12.The program may be provided to the information processing device 10 viaa communication network such as the Internet or may be provided in sucha way as to be stored in a computer-readable information storage mediumsuch as an optical disk.

The sensor value acquisition section 51 acquires sets of detectionvalues of the individual sensors at the times of periodic transmissionsthereof from the controller device 30. Specifically, the sensor valueacquisition section 51 acquires sets of a detection value of theproximity sensor 36 and detection values of the proximity sensors 33-1to 33-4 from the controller device 30 at predetermined time intervals.Additionally, in the case where the user is operating the trigger button35, the sensor value acquisition section 51 also acquires the numericvalue of the push-in amount of the trigger button 35.

Through the use of the detection values of the proximity sensors 33-1 to33-4 which have been acquired by the sensor value acquisition section51, the finger state identification section 52 identifies a state of theuser's thumb at the times of the detections of the detection values.Specifically, the finger state identification section 52 calculatesrotation angles of the individual joints, which define the movement ofthe user's thumb, on the basis of the detection values of the proximitysensors 33.

The thumb changes its posture according to the movements of first,second, and third joints as counted from the tip of the thumb. In thefollowing, these joints will be called an IP joint, an MCP joint, and aCMC joint in the order mentioned above. This calling method is a methodconforming to anatomical joint names used in a paper “MRI-based skeletalhand movement model” (Georg Stillfried, Ulrich Hillenbrand, MarcusSettles and Patrick van der Smagt), etc. Among these joints, the IPjoint and the MCP joint which are intermediate joins of the fingerrotate in one pair of opposite directions in which both the IP joint andthe MCP joint are brought close to/away from the palm of the hand inconjunction with bending/stretching of the finger. In the following, therotation angle of the IP joint will be referred to as a pitch angle θpi,and the rotation angle of the MCP joint will be referred to as a pitchangle θpm.

Further, the CMC joint has a structure capable of moving in variousdirections to allow the direction of the entire thumb to vary. In thefollowing, a rotation angle of the CMC joint in the same pair ofopposite directions as the pair of opposite directions of the rotationangle of each of the IP joint and the MCP joint will be referred to as apitch angle θpc, and rotation angles of the CMC joint in two pairs ofopposite directions which are each orthogonal to the above pair ofopposite directions of the rotation angle of each of the IP joint andthe MCP joint will be referred to as a yaw angle θyc and a roll angleθrc. The posture of the entire thumb is identified by these fiverotation angles of the three joints. For this reason, the finger stateidentification section 52 calculates the values of the five rotationangles on the basis of the detection values of the four proximitysensors 33. FIG. 6 schematically illustrates pairs of opposite rotationdirections of the rotation angles calculated by the finger stateidentification section 52.

In the following, the detection values of the individual proximitysensors 33-1 to 33-4 will be denoted by S1 to S4. These detection valuesindicate distances from the individual proximity sensors 33 to the tipof the thumb. Note that it is assumed that the nearer a detecteddistance is, the larger a detection value is. Further, a total value ofthe individual detection values, which is calculated using the followingformula, will be denoted by Ts.

Ts=S1+S2+S3+S4

At this time, percentage values indicating the ratios of the individualdetection values relative to the total value can be calculated asfollows.

S1r=S1/Ts

S2r=S2/Ts

S3r=S3/Ts

S4r=S4/Ts

The finger state identification section 52 calculates the rotationangles θyc and θrc of the CMC joint, by using the percentage values S1rto S4r, by means of the following formulas.

θyc=YC1·S1r+YC2·S2r+YC3·S3r+YC4·S4r

θrc=RC1·S1r+RC2·S2r+RC3·S3r+RC4·S4r

Here, each of YC1 to YC4 and RC1 to RC4 is a weighting coefficient thatis set in advance for a corresponding one of the percentage values inview of a movable range of the thumb. The values of the yaw angle andthe roll angle, which are calculated using these formulas, are valueseach proportional to a weighting average of the four percentage values.

Further, the pitch angles θpi, θpm, and θpc of the individual joints arevalues that vary in conjunction with distances from the thumb to thesensor region 34 and can be calculated using the detection values S1 toS4 of the individual sensors by means of the following formulas.

θpi=PI1·S1+PI2·S2+PI3·S3+PI4·S4

θpm=PM1·S1+PM2·S2+PM3·S3+PM4·S4

θpc=PC1·S1+PC2·S2+PC3·S3+PC4·S4

Each of PI1 to PI4, PM1 to PM4, and PC1 to PC4 in the above formulas isa weighting coefficient that is set in advance for a corresponding oneof the detection values in view of the movable range of the thumb. Thevalues of the pitch angles, which are calculated using these formulas,are values each proportional to a weighting average of the detectionvalues of the four sensors.

The finger state identification section 52 calculates the values of thefive rotation angles by means of the above formulas. These values makeit possible to identify in what kind of posture the thumb is in itsentirety at the times of the detections by the proximity sensors 33. Inthis way, the finger state identification section 52 can calculate therotation angles of the individual joints, by using the detection valuesof the individual sensors or the percentage values relative to the totalvalue thereof, by means of simple calculations by one-dimensionalformulas.

Further, the finger state identification section 52 identifies a stateof the user's index finger, by using the detection value of theproximity sensor 36. Specifically, the finger state identificationsection 52 estimates to what degree the user's index finger is benttoward the palm on the basis of the detection value of the proximitysensor 36 and outputs the result of the estimation. Specifically, it isestimated that the nearer the proximity sensor 36 the user's indexfinger is located, to the greater degree the user is bending the indexfinger toward his or her palm. Thus, the finger state identificationsection 52 calculates a bending amount of the user's index finger fromthe detection value of the proximity sensor 36 according to a conversiontable or a conversion formula that is prepared in advance.

Further, in the present embodiment, in the case where an operation inputfor the trigger button 35 has been detected, the finger stateidentification section 52 neglects (rejects) the detection value of theproximity sensor 36 and identifies a state of the user's index fingerusing the push-in amount of the trigger button 35. The operation inputfor the trigger button 35 involves a mechanical movement of the button,and thus, it is supposed that information regarding an operation amountfor the mechanical movement has higher reliability than informationregarding the detection result of the proximity sensor 36. For thisreason, in the case where the operation input for the trigger button 35is available, the finger state identification section 52 uses theoperation input therefor in preference to the detection result of theproximity sensor 36, and thereby is capable of accurately identifyingthe state of the user's index finger. Further, by using the triggerbutton 35 in the detection of the bending amount of the user's fingermakes it possible to present to the user a feeling in a case where theuser touches an object existing in a virtual space, or any other similarcase.

Note that even in the case where the operation amount (push-in amount)of the trigger button 35 is used, the finger state identificationsection 52 can calculate the bending amount of the user's index finger,by using a conversion table or a conversion formula, that is given inadvance similarly to the case where the detection value of the proximitysensor 36 is used. In this case, the conversion table or the conversionformula is determined in such a way that a bending amount of the fingerwhich is calculated on the basis of the operation amount of the triggerbutton 35, in the case where an operation amount of the trigger button35 is equal to zero coincides with a bending amount of the finger whichis calculated on the basis of the detection result of the proximitysensor 36, in the case where a distance detected by the proximity sensor36 is equal to zero. This method makes it possible to seamlesslyidentify the state of the index finger from a state in which the user isreleasing the index finger from the trigger button 35 to a state inwhich the user is pushing the trigger button 35.

The video display control section 53 draws a video that is to bepresented to the user and outputs the video to the display device 40. Inthe present embodiment, the video display control section 53 isconfigured to draw a video indicating a condition inside a virtual spacein which various objects are disposed. In particular, the video displaycontrol section 53 is configured to dispose, inside the virtual space, ahand object that varies in conjunction with the bending/stretching stateof the user's finger which is identified by the finger stateidentification section 52 and draw the state of the hand object. Thisconfiguration makes it possible to, when the user performs an operationof bending, stretching, or opening the finger of his or her hand onwhich the controller device 30 is worn, operate the hand object insidethe virtual space in the same way as that for the movement of thefinger. Thus, the user can experience a situation as if, in thesituation, the user were directly touching or grasping another objectinside the virtual space with his or her hand.

As described above, the controller device 30 according to the embodimentof the present invention is configured to dispose the proximity sensors33 along the curved surface corresponding to the movement trajectory ofthe user's thumb, and thus is capable of accurately identifying thestate of the thumb using a relatively small number of the proximitysensors 33.

Further, the controller device 30 is capable of accurately identifyingthe state of the user's index finger by combining the proximity sensor36 and the trigger button 35.

Note that embodiments of the present invention are not limited to theabove-described embodiment. Specifically, the shape of the controllerdevice 30 and the arrangement positions of the proximity sensors 33,which are illustrated in the above description, are just examples, and adifferent shape and a different arrangement that bring about similareffects may be employed.

Further, in the above description, all of the rotation angles of theindividual joints constituting the thumb have been calculated on thebasis of the detection values of the proximity sensors 33, but a methodmay employed in which only partial parameters are calculated and theposture of the thumb is determined on the basis the values of thecalculated parameters. Further, the calculation formulas for calculatingthe rotation angles may be different from those described above.

Further, in the above description, the state of the index finger hasbeen identified by combining the button whose push-in amount isdetectable and the proximity sensor, but even for a finger other thanthe index finger, its state may be identified in a similar method.

Further, at least part of the processes having been configured to beperformed by the information processing device 10 in the abovedescription may be performed by the control circuit 37 inside thecontroller device 30. For example, the control circuit 37 may beconfigured to calculate the values of the rotation angles of theindividual joints, on the basis of the detection result of the proximitysensors 33, and transmit the calculated values to the informationprocessing device 10 as numeric values representing the state of theuser's thumb.

REFERENCE SIGNS LIST

1 Information processing system, 10 Information processing device, 11Control unit, 12 Storage unit, 13 Interface unit, 30 Controller device,31 Main body, 32 Fixture, 33 Proximity sensor, 34 Sensor region, 35Trigger button, 36 Proximity sensor, 37 Control circuit, 40 Displaydevice, 51 Sensor value acquisition section, 52 Finger stateidentification section, 53 Video display control section.

1. A controller device configured to be worn on a hand of a user, comprising: a plurality of sensors that, in a state in which the user wears the controller device on the hand, are disposed so as to be arranged along a movement direction of a tip of a thumb of the user on a virtual curved surface being projected toward a position of the thumb of the user, wherein the position of the thumb of the user is detected by using detection values of the plurality of individual sensors.
 2. The controller device according to claim 1, wherein, in the state in which the user wears the controller device on the hand, the plurality of sensors are disposed so as to be arranged along a virtual curved line being projected toward a side opposite a base side of the thumb of the user and extending along the movement direction of the tip of the thumb of the user.
 3. An information processing device comprising: an acquisition section that acquires, from a controller device configured to be worn on a hand of a user and including a plurality of sensors that, in a state in which the user wears the controller device on the hand, are disposed along a virtual curved surface being projected toward a thumb of the user and detect a position of the thumb of the user, detection values of the plurality of individual sensors; and a state identification section that identifies a state of the thumb of the user by calculating a rotation angle of at least one or more joints of the thumb on a basis of the acquired detection values.
 4. The information processing device according to claim 3, wherein the controller device includes a detection sensor that detects a position of a target finger of the user within a predetermined detection range, and an operation button operated by the target finger, the acquisition section acquires a detection result of the detection sensor, the state identification section identifies a state of the target finger on a basis of the detection result of the detection sensor, and, when the operation button is pushed, the state identification section rejects the detection result of the detection sensor and identifies the state of the target finger on a basis of an operation amount detected by the operation button.
 5. An information processing method comprising: acquiring, from a controller device configured to be worn on a hand of a user and including a plurality of sensors that, in a state in which the user wears the controller device on the hand, are disposed along a virtual curved surface being projected toward a thumb of the user and detect a position of the thumb of the user, detection values of the plurality of individual sensors; and identifying a state of the thumb of the user by calculating a rotation angle of at least one or more joints of the thumb on a basis of the acquired detection values.
 6. A non-transitory, computer readable storage medium containing a program, which when executed by a computer, causes the computer to perform an information processing method by carrying out actions, comprising: acquiring, from a controller device configured to be worn on a hand of a user and including a plurality of sensors that, in a state in which the user wears the controller device on the hand, are disposed along a virtual curved surface being projected toward a thumb of the user and detect a position of the thumb of the user, detection values of the plurality of individual sensors; and identifying a state of the thumb of the user by calculating a rotation angle of at least one or more joints of the thumb on a basis of the acquired detection values. 